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Joint Access Point Selection and Power Allocation for Uplink Wireless Networks 

Mingyi Hong, Alfredo Garcia, Jorge Barrera and Stephen G. Wilson 

Abstract 

We consider the distributed uplink resource allocation problem in a multi-carrier wireless network 
with multiple access points (APs). Each mobile user can optimize its own transmission rate by selecting 
a suitable AP and by controlling its transmit power Our objective is to devise suitable algorithms by 
which mobile users can jointly perform these tasks in a distributed manner Our approach relies on a 
game theoretic formulation of the joint power control and AP selection problem. In the proposed game, 
each user is a player with an associated strategy containing a discrete variable (the AP selection decision) 
and a continuous vector (the power allocation among multiple channels). We provide characterizations 
of the Nash Equilibrium of the proposed game, and present a set of novel algorithms that allow the users 
to efficiently optimize their rates. Finally, we study the properties of the proposed algorithms as well as 
their performance via extensive simulations. 

I. Introduction 



In this paper we consider the joint access point (AP) selection and power allocation problem in a 
. wireless network with multiple APs and mobile users (MUs). The APs operate on non-overlapping 

G^■ 

I spectrum bands, each of which is divided into parallel channels. Each MU's objective is to find a suitable 

■ AP selection, followed by a vectorial power optimization. 

^ I The problem of joint AP selection and power control belongs to the category of cross-layer resource 

allocation in wireless communications. Such problem is important in many practical networks. For ex- 
ample, in the IEEE 802.22 cognitive radio Wireless Regional Area Network (WRAN) HI, a geographical 
region may be served by multiple service providers (SPs), or by multiple APs installed by a single 
SP 131. The users transmit by sharing the spectrum offered by a SP/AP. Moreover the users enjoy the 
flexibility of being able to dynamically select the less congested SP/AP. Another example in which 
such joint optimization plays an important role is the heterogenous networks Q, [Si, in which different 
technologies such as Wi-Fi, 3G, LTE or WiMAX are available for the same region. The MUs can choose 
from one of these technologies for communication, and they can switch between different technologies 
to avoid congestion (the so called "vertical handoff, see lH). As suggested in ^ and Q, compared 
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with the traditional closest AP assignment strategy, it is generally beneficial, in terms of the system-wide 
performance and the individual utilities, to include the AP association as the users' decision variable 
whenever possible. 

Distributed strategies for resource allocation in multi-carrier or single-carrier network with a single 
AP have been extensively studied. In |[8l, the authors consider the uplink power control problem in a 
fading multiple access channel (MAC). A water-filling (WF) game is devised in which the users compete 
with each other to maximize their individual rates. The authors show that the MAC channel capacity is 
achieved when all the users have perfect knowledge of all other users' channel state information (CSI). 
Such assumption, however, is unrealistic in a distributed setting. Reference 191 proposes a distributed 
power allocation scheme for uplink OFDM systems where the channel state is modeled to having only 
discrete levels. Notably, this scheme only requires that each user has the knowledge of its own CSI, which 
greatly reduces the signaling overhead. Reference ifTOl considers a generalization of a multi-carrier MAC 
channel, and proposes a distributed iterative water-filling (IWF) algorithm to compute the maximum sum 
capacity of the system. 

The problem of joint AP selection and power control has been addressed in single-carrier cellular 
networks as well. References ifTTl and |[T2l are early works that consider this problem in an uplink 
spread spectrum cellular network. The objectives are to find an AP selection and power allocation tuple 
that minimizes the total transmit powers while maintaining the users' quality of service requirements. 
The authors of m and Q cast a similar problem (with an objective to minimize the individual cost) 
into a game theoretical framework, and propose algorithms to find the Nash Equilibrium (NE) of their 
respective games. All of these papers consider the scalar power allocation problem. The work presented 
in this paper differs significantly in that a more difficult vector power allocation problem is considered, 
in which the users can potentially use all the channels that belong to a particular AP. 

There are a few recent works addressing related joint power control and band selection or channel 
selection problems in multicarrier networks. Reference |[T3l proposes to use game theory to address 
the problem of distributed energy-efficient channel selection and power control in an uplink multi-carrier 
CDMA system. Again the final solution mandates that the users choose a single optimum channel as well 
as a scalar power level to transmit on the selected channel. Reference |[3l considers the uplink vector 
channel power allocation and spectrum band selection problem in a cognitive network with multiple 
service providers. The users can select the size of the spectrum as well as the amount of power for 
transmission. However, the authors avoid the difficult combinatorial aspect of the problem by assuming 
that the users are able to connect to multiple service providers at the same time. Such assumption 
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may induce considerable signaling overhead on the network side as well as hardware implementation 
complexity on the user device Q. 

To the best of our knowledge, this is the first work that proposes distributed algorithms to deal with 
joint AP selection and power allocation problem in a multi-channel multi-AP network. We first consider 
the single AP configuration, and formulate the uplink power control problem into a non-cooperative game, 
in which the MUs attempt to maximize their transmission rates. An iterative algorithm is then proposed 
that enables the MUs to reach the equilibrium solutions in a distributed fashion. We then incorporate the 
AP selection aspect of the problem into the game. In this more general case, each user can select not only 
its power allocation, but its AP association as well. Although non-cooperative game theory has recently 
been extensively applied to solve resource allocation problems in wireless networks (e.g., ifTSl - lfTTl ). the 
considered game is significantly more complex due to its mixed-integer nature. We analyze the NE of 
the game, and develop a suite of algorithms that enable the MUs to identify equilibrium solutions in a 
distributed fashion. 

This paper is organized as follows. In Section |lll we consider the power allocation problem in a single 
AP network. From Section JII] to Section |Vll we discuss the problem of joint AP selection and power 
allocation. We introduce a game theoretic formulation of the problem and propose algorithms to reach 
the NE of the game. In Section IVlIl we show the numerical results. The paper concludes in Section [Villi 

Notations: We use bold faced characters to denote vectors. We use x[i] to denote the ith element of 
vector X. We use x_j to denote the vector [x[l], • • • , x[? — 1], x[i + 1], • • • x[A^]]. We use ej to denote a 
unit vector with all entries except for the jth entry, which takes the value 1; use e to denote the all 1 
vector. We use [y, x_j] to denote a vector x with its ith element replaced by y. The key notations used 
in this paper are listed in Table Jl 





Total interference for MU i on channel k 
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User i's rate when connected with AP w 




The set of all users with AP w 



TABLE I 

The Summary of Notations 



'in WLAN literature, such network is also referred to as "multi-homing" network, see 1141 and the reference therein. 



July 19, 2012 DRAFT 



4 



II. Optimal Power Allocation in a Single AP Network 

In this section, we briefly consider the simpler network configuration with a single AP. We provide 
insights into the relationship between optimal and equilibrium power control strategies of the users. We 
study algorithms whose convergence properties will be useful in our subsequent study of the multiple 
AP configuration. 

A. System Model 

Consider a network with a single AP. The MUs are indexed by a set A/" = {1, 2, • • • , N}. Normalize 
the available bandwidth to 1, and divide it into K channels. Define the set of available channels as 
/C^{1,2,--- ,K}. 

Let denote the signal transmitted by MU i on channel k; let = E[|3;^p] denote the transmit 
power of MU i on channel k. Let ~ CN{Q,n^) denote the white complex Gaussian thermal noise 
experienced at the receiver of AP with mean zero and variance n!'. Let denote the channel coefficient 
between MU i and the AP on channel k. The signal received at the AP on channel k, denoted by y^, can 
then be expressed as: = Xli^i ^^^^ + ■ Let Pi = \p}-, - ■ ■ ,pf] ^ be MU i's transmit power profile; 
letp^[ 

Pi ) ■ ■ ■ ) Piv] ^ be the system power profile. Define Pi to be MU i's maximum allowable transmit 
power. The set of feasible transmit power vectors for MU i is Vi = |pi : Pi > 0, ^^^^pf < 

Assume that the AP is equipped with single-user receivers, which treat other MUs' signals as noise 
when decoding a specific MU's message. This assumption allows for implementation of low-complexity 
receivers at the AP, and it is generally accepted in designing distributed algorithm in the MAC, see l8l , 
lUTl . Under this assumption, the MU i's achievable rate can be expressed as ifTSl : 

Clearly the rate Ri{-) given above represents the information theoretical rate that can be achieved only 
when using Gaussian signaling. This rate serves as the performance upper bound for any practically 
achievable rates. It is widely used for the design and analysis of resource allocation and spectrum 
management schemes in wireless networks, see e.g., ||8l, ifTTl . |[T9ll . 

Another important assumption is that each MU i G knows its own channel coefficients {hf}keK, 
and the sum of noise plus interference + each channel k. These pieces of information 

can be fed back by the APs |[T3l . 

Assume that time is slotted, and there is network-wide slot synchronization. Such synchronization can 
be made possible by equipping the users with GPS devices. The MUs can adjust their power allocation 
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in a slot by slot basis. The task for each MU is then to compute its optimal power allocation policy in 
a distributed manner. In the following subsection, we formulate such problem into a game-theoretical 
framework. 

B. A Non- Cooperative Game Formulation 

To facilitate the development of a distributed algorithm, we model each MU as a selfish agent, and 
its objective is to maximize its own rate. More specifically, MU i is interested in solving the following 
optimization problem: 

K 



max 



where = '^j^.iPj\hj\'^ is the total interference MU i experiences on channel k. The solution to this 
problem, denoted as p* is the well-known single-user WF solution, which is a function of {/f}f=i HI: 



- V fc G /C (3) 



where aj > is the dual variable for the power constraint. Let Ij = {/f be the interference 
experienced by MU i on all channels. We can define the vector WF operator as: 

^,{h)^[<fUmr-- ,<I'f(/f)]. (4) 

We introduce a non-cooperative power control game where i) the players are the MUs; ii) the utility 
of each player is its achievable rate; Hi) the strategy of each player is its power profile. We denote this 
game as G = {M, V, {-Ri(-)}i6AA}. where V = Wi^j^fVi is the joint feasible region of all MUs. The NE 
of game G is the strategies satisfying f20l : 

p* G arg max i?j(pj, plj \f i E J\f. (5) 

Intuitively, a NE of the game is a stable point of the system in which no player has the incentive to 
deviate from its strategy, given the strategies of all other players. To analyze the NE, let us introduce the 
potential function P : P — M as follows: 

We can readily observe that for any pj and pi G Vi and for fixed p_i, the following identity is true 

Ri{Pi,P-i) - RiiPi, P-i) = P{Pi, P-i) - P{pi,p-i). (7) 
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Due to the property ([7]), the game G is referred to as a cardinal potential game. Note further that the 
potential function -P(p) is concave in p. The following theorem is a classical result in game theory (see, 



Theorem 1: A potential game with concave potential and compact action spaces admits at least one 
pure-strategy NE. Moreover, a feasible strategy is a NE of the game if and only if it maximizes the 
potential function. 

In light of the above theorem, we immediately have the following corollary. 
Corollary I: p* is a NE of the game G if and only if 



We note that the potential game formulation for a single AP network is not entirely new. It can be easily 
generalized from the existing results such as |[22l . The main purpose of going through this formulation 
in detail is to facilitate our new formulation for the multiple AP setting in later sections. 

Interestingly, the maximum value of the potential function Q equals to the maximum achievable sum 
rate of this A^-user K-channel network. Such equivalence can be derived by comparing the expression 
for the sum capacity of this multichannel MAC with the potential Q. This observation indicates that at 
a NE of the game G, the MUs' transmission strategy is capacity achieving. However, in general the NE 
is still inefficient, meaning that the sum of the individual MUs' rates is less than the maximum sum rate 
of the system, as only suboptimal single-user receivers are used at the AP 

One exception to such inefficiency is when the NE represents an FDMA strategy, in which there is 
at most one MU transmitting on each channel. Obviously single-user receiver is optimal in this case, 
as there is no multiuser interference on any channel. Reference ||23l shows that when the number of 
channels tends to infinity, any optimal transmission strategy is FDMA for a multichannel MAC channel. 
In our context, this result says when K becomes large, any NE of the game G represents an efficient 
FDMA solution. 

C. The Proposed Algorithms and Convergence 

We have established that finding the NE of the game G is equivalent to finding p* € arg maxpgp -P(p), 
which is a convex problem and can be solved in a centrahzed way. However, when the MUs are selfish 
and uncoordinated, it is not clear how to find such NE point in a distributed fashion. 

^ In general, one needs to have both optimal transmission and optimal receiving strategies to achieve the MAC capacity, 
where the optimal receiving strategy is to perform the successive interference cancelation, see 1101 , |18| . 

July 19, 2012 DRAFT 



e.g., my 




(8) 



7 



Reference lITOl proposes a distributed algorithm named sequential iterative water filling (S-IWF) to 
compute the solution of the problem maxpgp P{p)- In iteration t of the S-IWF, a single MU i updates 
its power by pf^^^ = ^i(lf^), while all other MUs keep their power allocations fixed. Although simple 
in form, this algorithm requires additional overhead in order to implement the sequential update. In 
addition, when the number of MUs becomes large, such sequential algorithm typically converges slowly. 
To overcome these drawbacks, we propose an alternative algorithm that allows the MUs to update their 
power allocation simultaneously. 

Averaged Iterative- Water Filling Algorithm (A-IWF): 
At each iteration t, the MUs do the following. 

1) Calculate the WF power allocation \/ i eJ\f. 

2) Adjust their power profiles simultaneously according to: 

pf^ = (1 -a(*))pf +a«*,(lf ), V z e (9) 
where the sequence {aW}^i satisfies aW G (0,1) and : 

T T 

lim Va^*^ = oo, lim Vfa^*^^ < oo. (10) 

t=l t=l 

The convergence property of the A-IWF algorithms is stated in the following proposition, the proof of 
which can be found in Appendix IIX-AI 

Proposition I: Starting from any initial power p*-*^) G V, the A-IWF algorithm converges to a NE of 
game G. 

We note that a similar algorithm has been recently proposed in |[24l for computing equilibria for a power 
control game in the interference channel. However, the convergence of the algorithm proposed in Il24l 
requires some contraction properties of the WF operator, which in turn requires that the interference 
should be weak among the users. Unfortunately, such stringent weak interference condition cannot be 
satisfied in our current setting (see 1251 for an argument). For this reason, completely different techniques 
are used to prove the convergence of the A-IWF algorithm in the present paper. 

III. The Joint AP Selection and Power Control 
In this section, we begin our discussion on the more general network in the presence of multiple APs. 

A. System Model 

Let us again index the MUs and the channels by the sets A/" = {1, 2, • • • , A^} and /C = {1, 2, • • • , A'}, 
respectively. Introduce the set W = {1,2,- •• ,W} to index the APs. Again, we normalize the total 
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bandwidth to 1. Each AP w & W is assigned with a subset of channels /C^j C /C. We focus on the 
uplink scenario where each MU transmits to a single AP. The following are our main assumptions of the 
network. 

A-1) Each MU i is able to associate to any AP. 

A-2) The APs operate on non-overlapping portions of the available spectrum. 
A-3) Each AP is equipped with single-user receivers. 

Assumption A-1) is made merely for ease of presentation and simplicity of notation. To account for the 
case that each MU has its own set of candidate APs, one would simply need to introduce a user-indexed 
subset of the APs as the feasible set of AP choices for each MU. Assumption A-2) is commonly used 
when considering AP association problems in WLAN (e.g., see |[T4l ). or the vertical handoff problems in 
heterogenous networks (e.g., see (5]), or the spectrum sharing problems in cognitive network with multiple 
service providers (e.g., see |[3j). Assumption A-2) impHes that ICuj H = 0; y w q,k,q £ W. 

We now list the additional notations needed for our analysis: 

• wl"^} keK^. and {n'^}k£K,, denote respectively, the power gains from MU i to AP w on all 
channels and the thermal noise powers on all channels for AP w. 

• a G denotes the association profile in the network, i.e. a[i] = w indicates that MU i is associated 
to AP w. 

• Mw — {i '■ a[i] = w} denotes the set of MUs that are associated with AP w. By the restriction that 
each user can choose a single AP, {J\fw}wew is a partition of J\f. 

• p^^ represents the amount of power MU i transmits on channel k when it is associated with AP w. 

• Pi,w — {Piw}k£K ^^^^^^^ t^*^ power profile of MU i when a[i] = w; — {Pi,w}i^j^ denotes 
the power profiles of all MUs associated with AP w. 

• P-i^w — {Pj,w} j:jyti,a[j]=w denotes the power profiles of all the MUs other than i that is associated 
with AP w. 

• — Y^j:a,[i]=w,j^i\h),w?P),w denotes the interference for MU i on channel k; = {-^f j^^g;^^ is 
the set of all interferences on all channels of AP w. 

'■YjkeK^ pIw < Pi,w > 0, V fc e /Cu,| denotes MU i's feasible power allocation when 
it is associated with AP w. 
When associated with AP w, MU i's uplink transmission rate can be expressed as 



July 19, 2012 



DRAFT 



= -^IZ^°s(i+ T/tT ) =i?*(p«,»,i^,»;w')- (12) 

Fixing the association profile a and all the other MUs' power allocation p_j,«,, we use to 
denote the vector WF solution to MU i's power optimization problem maxp^ ^gjr^ ^ Ri{Tpi,w,'P-i,w] w)- 

B. A Non-Cooperative Game Formulation 

We consider the non-cooperative game in which each MU selects an AP and a power allocation over the 
channels at the selected AP. This game can be formally described as follows: i) the MUs are the players; 
ii) each MU's strategy space is Xi — UujeW i^' MU's utility is i?j(pj^^, p„j^^; w). We 
denote this game as = {M ,{xi]iej\f,{Ri}ieM}- We emphasize that the feasible strategy of a player 
contains a discrete variable and a continuous vector, which makes the game Q more complicated than 
most of the games considered in the network resource allocation literature. 

The NE of this game is the tuple |a*[i], p*a.[j]| ^ that satisfies the following set of equations: 

a*W,P*a-m) eargmax max i?j(pj,„, pl^ ^; w), V i. (13) 

We name the equilibrium profile a* a NE association profile, and p*(a*) = {p* i^j^ ^ power 
allocation profile. To avoid duplicated definitions, we name the tuple (a*,p*(a*)) a joint equilibrium 
profile (JEP) of the game Q (instead of a NE). It is clear from the above definitions that in a JEP, the 
system is stable in the sense that no MU has the incentive to deviate from either its AP association or 
its power allocation. 

C. Properties of the JEP 

In this section, we generalize the potential function for the power allocation game G to the game Q. 
We then prove that the JEP always exists for the game Q. 
We define the potential function for the game Q as follows. 

Definition 1: For a given tuple (a, p), define the potential fimction for the AP w as : 
P„(p„;a) = ^ J2 (log ("t + I] HJ^P-,^) - logni j . 
Then the potential function for the game Q is defined as: 

P(p;a) ^ J] P^(p„,;a). (14) 
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For a given association profile a G , define J-"^(a) = HieAT •^i,w as the joint feasible set for the MUs 
in the set Afw Let J-"(a) = n«;GW ^w{si). Our next result characterizes the system potential function. It is 
a straightforward consequence of Theorem [T] Corollary [T] and the observation that for a given association 
profile a, the activities of two different MUs i,j do not affect each other as long as a[i] ^ a[j]. 

Corollary 2: For a given a, a feasible p* € -7^(a) maximizes the potential function P(p;a) if and 
only if for all w € W, pj^ maximizes the per-AP potential function, 

p*^ e max P^(p^„;a), V u; G W. (15) 

Moreover, p^ is the NEfor a single AP power allocation game G, characterized by the potential function 
Pw {Pw ; a), (ind with Nyj as the set of players. 

We note that problem ( fTSl ) is precisely the single AP power allocation problem discussed in Section |lll 
Hence the power allocation p^ can be computed distributedly by the set of MUs Mw using the A-IWF 
or S-IWF algorithm. 

For a given a G , let <?(a) and P(a) denote the set of optimal solutions and the optimal objective 
value of the problem maxp^jr^^) P(p;a), respectively. Similarly, define Sw{3i) and P^(a) as the set of 
optimal solutions and the optimal objective value for problem ([TS] ). As discussed in Section JIJ each 
subproblem in ( fTSl ) is a convex problem hence Pu,(a) is unique. It follows that for a given a, -P(a) takes 
a single value as well. Therefore, P(a) can be viewed as a function of the association profile a. 

We emphasize that determining the existence of the JEP (which is a pure NE) for the game Q is by 
no means a trivial proposition. Due to the mixed-integer structure of the game Q, the standard results on 
the existence of the pure NE of either continuous or discrete games cannot be applied. Consequently, we 
have to explore the structure of the problem in proving the existence of JEP for the game Q. 

Theorem 2: The game Q always admits a JEP in pure strategies. An association profile a G arg maxa -P(a), 

along with a power allocation profile p = {pi,5[j]},.g^ G <f (a), constitute a JEP of the game Q. 

Proof: We prove this theorem by contradiction. Suppose a maximizes the system potential, but a 
is not a NE association profile. Then there must exist a MU i who prefers to switch from a[i] = u; to a 
different AP w ^ w. Define a new profile a = [u;, a_j]. Let p G <?(a) and p G <f(a). Suppose that all 
other MUs do not change their actions, then the maximum rate that MU i can obtain after switching to 
AP w is given by 

^» (p.^> p^; = ]^ L i«g — ^FT-^ \hn^' 

= ^^(Pi.ffiiPffi;^) - Pffi(pffi;a) (16) 
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where the vector is determined by: pj {5 = argmaxp^gj-^ _ i?j(pj, p^; tt;). The rate Ri{pi^w, Pw', w) 
is MU i's estimate of the maximum rate it can get if it were to switch to AP w. Let Ri{pi^w, p~i,is'-, w) 
denote the actual transmission rate for MU i in profile a. Similarly as in ([T6l ). it can be explicitly 
expressed as 

Rt{Vt,w,'P-i,w,w) /'5i(pffi;a) - Ps(p_j^^;a). (17) 

Because MU i prefers w, from the definition of the JEP ([T3]) it follows that its current rate must be 
strictly less than its estimated maximum rate: 

Ri{Pi,w3-i.w]w) < Ri{pi.{s,P{s;w). (18) 

Combining (fT6l ). (ITtI ) and ([TSl l we conclude that: 

Pw{Pw]^)^Pw{P-i,S)i^) < Pw{Pi,w:Pw]^) ~ Pw{Pw]^)- (19) 

Note that the term P{5(p_, {„; a) is equivalent to P^(p_j^{5;a) due to the equivalence of the following 
sets {j : j / i,a[j] = w} and {j : j ^ «,a[j] = w}. Due to the assumption that p G from 
Corollary |2] we must have that p^j maximizes the potential function of AP w for the given association 
profile a: p^ G argmaxp^gjr_(g) P{^;(p{5;a). Using the fact that the set of MUs associated with AP w 
under profile a is the same as the set of MUs associated with AP w under profile a excluding MU i, we 
must have p_j^{5 G -Fuj(a). Consequently, the following is true: 

P{5 (pa ; a) > P{s (p_j,i3 ; a) = P{5 (p.j^is ; a) (20) 

Similarly, we have that: 

Pffl(pffl;a) > Pffi(pi,ffi,Pffi;a). (21) 
Combining dlOll, (EB and (O, we have that: 

Pw {Pw ] ~ Pw (Ptu ; 3-) < PiB (pffi ; 3-) ~ Pw (pffi ; 
Rearranging terms, we obtain: 

Pe (pa ; a) + Pa (pffl ; a) < Pffi (pa ; a) + Pa {pa ; a) . (22) 
Finally, notice that (p^ ; a) = (p„, ; a) for all the APs w other than w and w. Then (l22l ) is equivalent 
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to: 



(23) 



which is further equivalent to: P{a) < P(a). This is a contradiction to the assumption that a G maxa -P(a). 
We conclude that a must be a NE association profile. Clearly, p is a NE power allocation profile. 



In this section, we introduce our first algorithm, referred to as the Joint Access Point Selection and 
Power Allocation (JASPA) algorithm, that allows the MUs in the network to compute the JEP in a 
distributed fashion. 

In the proposed algorithm, the AP selections and the power allocation adjustments take place over 
different time scales. While the APs are chosen in a relatively slow time scale, power allocations are 
made in a faster time scale. The properly chosen time scales enable the power control algorithm to 
reach an equilibrium before the AP selections are updated. Once the convergence of power allocation is 
achieved, the MUs attempt to find different APs to further increase their rates. Let us assume that each 
MU maintains a length M memory, which operates in a first in first out (FIFO) fashion. When MU i 
decides that its next best AP association should be w* , it records the decision by a 1^ x 1 best-reply vector 
hi : hi = e^*, and pushes this vector into its memory. In the next iteration, the actual AP association 
decisions for the MUs are made based on the elements in their respective memories. Let (p(*\a(*)) 
denote the power and association profile of iteration t. The proposed algorithm is detailed as follows. 

1) Initialization: Let t=0, MUs randomly choose their APs. 

2) Power Allocation: Fix a*^*', in each cell, the set of MUs Mw calculate the power allocations 
Pw^^^ G '?M;(a^*^), either by A-IWF or S-IWF algorithm. The process of reaching such intermediate 
equilibrium is referred to as an "inner loop". 

3) Find the Best AP: Each MU i communicates with all the APs in the network, obtains necessary 
information to find a set W^^^^^ of APs such that all w G IV,-*^^^ satisfies: 



Randomly select the AP w* G Wl . Set the best-reply vector b^^ ' = e^.. 
4) Update the Probability Vector: For each MU i, update aWxl probability vector pf^ according 



Consequently, (a, p) is a JEP. 



IV. The Proposed Algorithm 




(24) 
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to: 



/3^^J + ^(bf+i)-bf-'^)) if t>M 



/3. 



(t+i) 



,(1) 



if A/ > i > 
if i = 0. 



(25) 



Remove hf from the front of the memory if t > Af; push b|*^^^ into the end of the memory. 

5) Find the Next AP: Each MU i samples the AP index according to: a(*+^)[i] ~ multi(/3f '''^^), 
where multi(-) represents a multinomial distribution. 

6) Continue: If a(*+i) = a(*+i^™) for all m = 1, • • • , M, stop. Otherwise let t=t+l, and go to Step 

2). 

We are now ready to make several comments regarding to the proposed JASPA algorithm. 

Remark 1: It is crucial that each MU finally decides on choosing a single AP. Failing to do so results 
in system instability, in which the MUs switch AP association indefinitely. More precisely, it is preferable 
that for all i G M, lim^^oo (3f^ = e^* for some w* G W. 



i,w ' 



W). 



Remark 2: To perform step 3) of JASPA, MU i needs to compute its WF solution 
For such purpose, only the vector of total interference if^^^ is needed from AP w. This is precisely the 
necessary information needed for finding the set W^^~^^^ in step 3) of the algorithm. 



Shift into 
Memory 



b:+' = 



1 





w' + l 



W 



1^: 



Update 
Probability 
Vector 



b- 



/ 


1 




User Memory 



Shift Out of 
Memory 



Fig. 1. Graphical Illustration of Step 4 of the JASPA algorithm. 

Remark 3: In step 4) of JASPA, each MU's probability vector (3^ is updated. A graphical illustration 
of this step for user i is provided in Fig. [T] By following the update rule in (1251 ). the wl-th entry of /3j 
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will be at least jj over the next M iterations. This step combined with the randomization procedure in 
step 5) ensure that all of MU i's best AP in the most recent M iterations have the probability of at least 
jj of being selected as a'^*'^^^[f]. This is in contrast to a naive greedy strategy that always selects the 
best AP for the MUs in every iteration. In fact, such greedy algorithm may diverge while the proposed 

randomized selection algorithm guarantees convergence (see Theorem [3]) 

Remark 4: To avoid unnecessary overhead related to ending and establishing the connections, it is 
reasonable to assume that a selfish MU is willing to leave its current AP only if the new AP can offer 
significant improvement of the data rate. To model such behavior, we introduce a connection cost Ci > 
for each MU. Then the set yvf~^^^ should contain the APs that satisfy the following inequality 



From a system point of view, introducing these connection costs may improve the convergence speed 
of the algorithm, as the MUs are less willing to change associations. However it might also result in 
reduced system throughput. 

Remark 5: Suppose at time T + 1, the algorithm terminates with profile a*. Then a* must be a NE 
association profile. To see this, note that the algorithm stops when a* appears in M consecutive iterations. 
From the way that the actual AP association is generated, we see that during iterations (T— M+1, T+1), 
each MU i must prefer a*[i] at least once. More precisely, a* must be preferred by all the MUs in the 
system. It follows that a* is an equilibrium solution. 

The following theorem shows that the JASPA converges to a JEP globally regardless of the starting 
points or the realizations of the channel gains. See Appendix IIX-BI for proof. 

Theorem 3: When choosing M > N, the JASPA algorithm produces a sequence {(a(*\ p*-*^)}^]^ that 
converges to a JEP (a*,p*(a*)) with probability 1. 

V. Extensions to the JASPA Algorithm 

The JASPA algorithm presented in the previous section is "distributed" in the sense that the computation 
in each iteration can be performed locally by the MUs. However, it requires the MUs to jointly implement 
an intermediate power equilibrium p(*+^) between two AP selections a^*) and sS-^^^\ which entails 
significant coordination among the MUs. In this section, we propose two extensions of the JASPA 
algorithm that do not require the MUs to reach any intermediate equilibria. 

^Indeed, a simple example of the divergence of the greedy algorithm is in a 2 AP 2 user network, in which each AP has a 
single channel. Suppose the users' channels are all identical, and both users associate with the same AP at the beginning. Then 
each of them will always perceive the vacant AP as its "best AP". Each of them will then switch back and forth between the 
APs and will never stabilize. 



max 
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The first algorithm, named Se-JASPA, is a sequential version of the JASPA. It is detailed in Table Jll 



1) Initialization: Each MU randomly chooses a'^"'[i] and pf'lia)^^-^ 

2) Determine the Next AP Association: 

If it is MU IS turn to act, (e.g., {{t + l)mode7V} + 1 = i): 
2a) MU i finds a set that satisfies 

W-*^^^ = {w* ■■ argmaxtugw maxp.^gjr.^ i?(pi ujjplf^u')}. 
2b) MU i selects an AP by randomly picking a(*+^)[i] € IV-*'''^^ 
For other MUs j ^ i, a(*+i)[j] = a(*)[j]. 

3) Update the Power Allocation: 

Denote w* = a(*+i)[i]. MU i updates by p|*+j^' = 

For other MUs j ^ i, p^^'^ = pj-*!, ■ 

4) Continue: Let t=t+l, and go to Step 2) 

TABLE II 
The Se-JASPA Algorithm 



The Se-JASPA algorithm differs from the original JASPA algorithm in several important ways. Firstly, 
each MU i £ M does not need to record the history of its best-reply vectors {b^*^}t. It decides on its 
AP association greedily in step 2). Secondly, a MU i, after deciding a new AP a(*"^^)[i] = w*, does not 
need to go through the process of reaching an intermediate equilibrium. However, the MUs still need 
to be coordinated for the exact sequence of their update, because in each iteration only a single MU is 
allowed to act. As can be inferred by the sequential nature of this algorithm, when the number of MUs 
is large, the convergence becomes slow. 

An alternative simultaneous version of the algorithm (named Si- JASPA) is listed in Table Hill Differently 
from the Se-JASPA algorithm, it allows for all the users to update in each iteration. We note that in the 
algorithm, the variable Tj represents the duration that MU i has stayed in the current AP; the sequence 
of stepsizes {a^*^}^]^ is chosen according to (ITOl) . 

The structure of the Si-JASPA is almost the same as the JASPA, except that for each MU, after 
switching to a new AP, it does not need to go through the process of joint computation of the intermediate 
equilibrium. Instead, the MUs can make their AP decisions in each iteration of the algorithm. The level 
of coordination among the MUs required for this algorithm is minimum among all the three algorithms 
introduced so far. 

Although to this point there is no complete convergence results for the Se/Si-JASPA algorithms, our 
simulations suggest that they indeed converge. Moreover, the Si-JASPA usually converges faster than the 
Se-JASPA. 



July 19, 2012 



DRAFT 



16 



1) Initialization (t=0): 

Each MU i randomly chooses a^''^[i] and P^^(o)[j] 

2) Selection of the Best Reply Association: 

Each MU i computes bf following Step 3) of JASPA 

3) Update Probability Vector: 

Each MU i updates /3f according to dZSl l 

Push b^*"''^'' into the memory; 

Remove bj-*"^^' from the memory if t > M 

4) Determine the Next AP Association: 

Each MU i obtains a(*+i) [i] following Step 5) of JASPA 

5) Compute the Best Power Allocation: 
Let w* = a(*+i)H; 

Each MUs i calculates p*^,, by p,*^,, ; w*) 

6) Update the Duration of Stay: 

Each MU i maintains and updates a variable Tf. 
r 1 if< =^a(*)W 

' " \ T, + 1 if < =a(*)H 

7) Update the Power Allocation: 

Each MU i calculates pf^*'' as foUows: 

(t+i)^ / Plu.' ' ' if<^a«H 

P''-* \ (1 - a(^-))p(*) . + «(^-)p*^. if ^1 = a(*) W 

8) Continue: Let t=t+l, and go to Step 2) 

TABLE III 
The Si- JASPA Algorithm 



VL JASPA Based on Network-Wide Joint-Strategy 

In this section, an alternative algorithm with convergence guarantee is proposed. This algorithm allows 
the MUs, as in the Se/Si-JASPA, to jointly select their power profiles and AP associations without 
the need to reach any intermediate equilibria. We will see later that compared with the JASPA and 
its two variants introduced before, the algorithm studied in this section requires considerably different 
information/memory structure for the MUs and the APs. Among others, it requires that the MUs maintain 
in their memory the history of some network-wide joint strategy of all MUs. We henceforth name this 
algorithm the Joint-strategy JASPA (J-JASPA). 

A. The J-JASPA Algorithm 

The main idea of the J-JASPA algorithm is to let the MUs compute their strategies according to some 
system state that is randomly sampled from the history. This is different from the JASPA and Si- JASPA 
algorithm, in which the MUs compute their best strategies according to the current system state, but then 
perform their actual association by random sampling the history of the best strategies. To better introduce 
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the proposed algorithm, we first provide some definitions. 

• Let Nw'^ = |i : a(*)[i] = denotes the set of MUs that are associated with AP w in iteration t. 

• Let I(A/'^) = {li^w}i(zj^ be the joint interference profile of the set of MUs Mw- 

We then elaborate on the required memory structure. Let each MU i keep three different kinds of 
memories, each with length M and operates in a FIFO fashion. 

1) The first memory, denoted as aj, records MU i's associated APs in the last M iterations: {a^*) [«] 
i.e., ajm] = a('^-^^+") [i], for m = 1, • • • ,M. 

2) The second memory, denoted as Ij, records the MU i's interference levels in the last M iterations, 
Wyt=T-M+i ^^^^^ ^''^ ~ ^^^^^-ew- That is, Um] = for m = 1, • • • , M. 

3) The third memory, denoted as Rj, records MU i's rate in the last M iterations, {vflt) [j] ; a^*^ [i]^ | 



T 

t=T-M+l 

Each AP w is also required to keep track of some local quantities |3 related to the history of the 
MUs' behaviors. Suppose a subset of users QQ M has been associated with AP w at least once during 
iterations [0, T]. Let us define the time index t^(Q) = argmax^jA/'if'' = Q} as the most recent time 
index that Q appears in AP w. The following variables are required to be recorded by each AP w G W: 

• The local power profile p^(Q) = |p-^^^^| 

• The local interference profile I„,(Q) = jl*"^^^ j 

• The total number of times that Q has appeared in AP w: Tw{Q) = J2t<T '^{■^w'^ = 2}' where 
!{•} is the indicator function. 

The J-JASPA algorithm is stated as follows: 

1) Initialization: Let t = 0, each MU i randomly chooses a(^)[i] G W and P-°a(o)[j] £ J^i,am[i]- 

2) Update MU Memory: For each i G TV, obtain if^ from the APs. If t > M, Remove ajl], IJl] and 
Rj[l]. Push aW[i], lf\ and Ri ^p^*t)j.paW[i]^ into the end of a^, Ij and Rj, respectively. 

3) Update AP Memory: For each w G W, perform the following updates: 

r^(A/-W) = T^(A/-W) + l. 



4) Sample Memory: Let M = min{M, t}, and let e be the M x 1 all 1 vector. Each MU i samples its 



4 



Here, "local" means individual APs can gather these pieces of information without the need to communicate with other APs. 
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memories by: 

indcx;*^ ^ multi | — x e | ; 

\M J 

Set af^ = ai(indexf^), if = li(indexf)) and R^^ = Ri(indexf ). 

5) The Best AP Association: Each MU i computes the set of best APs W* by the sampled variables 
d^\jf and R?: 

W* = S^w:^ max R, (p,- ^ , ; w) > i?f ^ | (J af^ . (26) 
Randomly pick w* e W*, and set a(*+^)[i] = w*. 

6) The Power Allocation: Each MU i switches to AP a(*+^)[i] = w*. MU i obtains Tu,'^{j\f^^^^), 

I< i^Lf'^), and p^. {J\fif'^) from AP <. 

If T^. (A/"^*^^^) > 1, then let t^. = T^. (A/"^*^^^) and a = a^*< \ Choose power according to 

p^P = (1 - S)p,^. (AA^J^^)) + S*. (l.(AAl^^)); <) . (27) 

Else randomly pick pf^P € -^i.w*- 

7) Continue: Let t = t + 1, go to step 2). 

Comparing with the JASPA and Si-JASPA algorithms, the J-JASPA algorithm possesses the following 
distinctive features. 

1) In J-JASPA, each MU calculates its best AP association according to a sampled historical network 
state, while in the JASPA and Si-JASPA, it calculates this quantity according to the current network 
state. 

2) The J-JASPA algorithm requires the APs to have memory. Each AP needs to record the local power 
allocation and interference profiles for all the different sets of MUs that have been associated with 
it in the previous iterations. No such requirement is imposed on all the previously introduced 
algorithms. 

3) The J-JASPA requires larger memory for the MUs for constructing a^, Ij and Rj. 

4) The J-JASPA requires extra communications between the MUs and the APs. Such overhead mainly 
comes from step 6), in which the MUs retrieve information stored in the APs for their power 
updates. 

B. The convergence of the J-JASPA algorithm 

In this subsection, we show that the J-JASPA algorithm converges to a JEP The proofs of the claims 
made in this subsection can be found in Appendix IIX-CI to IIX-EI 
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Let a set A consist of association profiles that appear infinitely often in the sequence {a^*)}^]^. We first 
provide a result related to the power allocation along an infinite subsequence in which a ^ A appears. 

Proposition 2: Choose a £ A. Let {tn}^^i be a subsequence such that {t„ : a^*"-* = a}. Then for all 
w, lim„_5.oo Pw"^ = Pw' which is an optimizer of the problem maXp^ g jr^ (g) P^(p^; a). Furthermore, we 
have that lim„^oo -Pw (^p^^""*; a^*")^ = Pu,(ei) and lim„_^.oo -P (p^*"^ a^*")) = P(a). 

We need the following definitions to proceed. Let a^*) be the profile sampled from the memory by the 
MUs in step 4) at time t: a^^\i\ = af\ V i. For a specific a, let the subsequence {injJ^Li ^'e the time 
instances that a appears and is immediately sampled by all the MUs, i.e., |i„ : a*^*") = a and a^*") = a|. 
Note that if a^*) = a, then according to step 3)-step 4), with non-zero probability, a^*) = a. Thus, if 

a S ^, then {in} is an infinite sequence. 

Define R* (ifl^; u?j = maxp^ ^^jr^ ^ Ri ^pj I^*^; as the maximum rate MU i can achieve in AP 
w based on the interference if^. Define MU i's best association set as: 

(lf\a(*)w) ^ [u, : R: {^-w) > Rr^}^^] (28) 

where Rf^ is the sampled rate given in step 4). From step 5) of the J-JASPA algorithm, all w G 
Bi (lf\a^^^[ifj has non-zero probability to be the serving AP for MU i in iteration t + 1. Let I* (a) = 
lim„_j.oo I^^*"-'. Due to Proposition |2j and the fact that a appears infinitely often, such limit is well defined. 
Next we provide an asymptotic characterization of the best association set defined above. 

Proposition 3: For a specific MU i and a system association profile a £ A, suppose there exists a 
w 7^ a[i] such that w £ Bi (I*(a), a[i]), i.e., MU i has the incentive to move to a different AP in the 
limit. Then there exists a large enough constant N*(a) such that for all n > A^*(a), we have: 

(lf"\a(*")H). (29) 

In words, when a profile a appears infinitely often, and suppose in the limit, when a is sampled, a MU 
i prefers w / a[i]. Then it must prefer w in every time instance t„ when n is large enough. Now we are 
ready to provide the main convergence result for the J-JASPA algorithm. 

Theorem 4: When choosing M > N, the J-JASPA algorithm converges to a JEP with probability 1. 

VII. Simulation Results 

In this section, we demonstrate the performance of the JASPA algorithm and its three variants discussed 
in this work. The following simulation setting is considered. Multiple MUs and APs are randomly placed 
in a 10 meter by 10 meter area. We use „, to denote the distance between MU i and AP w. Unless 
otherwise noted, the channel gains between MU i and AP w are generated independently from an 
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exponential distribution with mean (i.e., is assumed to have Rayleigh distribution). Pre-assign 

the available channels equally to different APs. Throughout, a snapshot of the network refers to the 
network with fixed (but randomly generated as above) AP, MU locations and channel gains. The length 
of the individual memory is set as M = 10. For ease of comparison, when we use the JASPA algorithm 
with connection cost, we set all the MUs' connection cost {ci}i^_^ to be identical. 

We first show the results related to the convergence properties, and then present the results related 
to the system throughput performance. Due to the space limit, for each experiment we show the results 
obtained by running either Si/Se-JASPA and J-JASPA, or those obtained by the JASPA. 

1 ) Convergence: Only the the results for Si/Se-JASPA and J-JASPA are shown in this subsection. We 
first consider a network with 20 MUs, 64 channels, and 4 APs. Fig. |2] shows the evolution of the system 
throughput as well as the system potential function generated by a typical run of the Se- JASPA, Si-JASPA, 
J-JASPA and Si-JASPA with connection cost q = 3 bit/sec V i G TV. We observe that the Si-JASPA with 
connection cost converges faster than Si-JASPA and Se-JASPA, while Se- JASPA converges very slowly. 
After convergence, the system throughput achieved by Si-JASPA with connection cost is smaller than 
those achieved by the other three algorithms. 




Iterations Iterations 



Fig. 2. Comparison of convergence speed by the Si/Se-JASPA, J-JASPA and Si-JASPA with connection costs. A'^ — 20, 
K = 64, Q = 4 and Ci — 3 bit/s. Left: evolution of system sum rate. Right: evolution of the system potential function. 

Fig. m shows the evolution of the AP selections made by the MUs in the network during a typical run 
of the Si-JASPA algorithm. We only show 3 out of 20 MUs (the selected MUs are labeled as MU 1, 2, 
3 for easy reference) in order not to make the figure overly crowded. Fig. [3] shows the corresponding 
evolution of the probability vectors {pf^}f^i for the three of the MUs selected in Fig. |4l It is clear 
that upon convergence, all the probability vectors converge to unit vectors. Fig. |5] evaluates the impact 
of the number of MUs N on the speed of convergence of different algorithms. When N becomes 
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Fig. 3. Convergence of and generated by a typical run of Si-JASPA. iV = 20, A" = 64 and Q = 4. 



large, the sequential version of the JASPA takes significantly longer time to converge than the other 
three simultaneous versions of the algorithm. Moreover, the J-JASPA exhibits faster convergence than 
the Si/Se-JASPA. It is also noteworthy to mention that including the connection costs indeed helps to 
accelerate the convergence for the algorithm. 




-B-Si-JASPA 
-*-Si-JASPA with Cost 3 
.G-Se-JASPA 
.?-J-JASPA 




Number of MUs in tile Network 



Fig. 4. Convergence of selected MUs' AP selection generated 
by a typical run of Si-JASPA. N = 20, K = 64 and Q = 4. 



Fig. 5. The averaged number of iterations required for 
the convergence for different algorithms. K — 64, Q — 4, 
and Ci = 3 bit/s. Each point in figure is an average of 100 
independent runs of the algorithms. 



2 ) System Throughput Performance: We evaluate the throughput achievable by the JEP computed by 
the JASPA. Such throughput performance is compared against a simple baseline algorithm that assigns the 
users to their closest AP in terms of actual distance. After assignment, the power allocation is computed 
using the A-IWF algorithm discussed in Section |II1 

We first consider a small network with 8 MUs, 64 channels and W G {1, 2, 3, 4} APs. We compare 
the performance of the JASPA algorithms to the maximum throughput that can be achieved. The maximum 
throughput for a snapshot of the network is calculated by an exhaustive search procedure: 1) for a given 
association profile, say a, calculate the maximum throughput (denoted by T(a)) by summing up the 
maximum achievable rate of all APs in the network; 2) enumerate all possible association profiles, and 
find T* = maXaT(a). 
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In Fig. [6l we see that the JASPA algorithm performs very well with little throughput loss, while the 
closest AP algorithm performs poorly. We then investigate the performance of larger networks with 30 
MUs, up to 16 APs and up to 128 channels. Fig. [7] shows the comparison of the performance of JASPA, 
JASPA with individual cost c, = 3 bit/sec and a = 5 bit/sec. 



-& JASPA 

JASPA with Cost 3 
-e- Closest AP 

---Maximum System Capacity 




Number of APs 



-B- JASPA 
■A-K-Connectivity 
-A- JASPA with Cost 3 
-0-JAPSAwith Cost 5 
-Q- Nearest Neighbour 




' 


*^ K=128 












t"^ 2 4 8 ie 

Number of APs 



Fig. 6. Comparison of the system throughput by different Fig. 7. Comparison of the system throughput. A*' = 30, 

algorithms. iV = 8, if = 64 and Q G {1,2,3,4}. Each K € {64,80,128} and Q G {1,2,4,8,16}. Each point in 

point in figure is obtained by running the algorithm on 100 figure is obtained by running the algorithm on 100 snapshots 

snapshots of the network. of the network. 

Due to the prohibitive computation time required, we are unable to obtain the maximum system 
throughput for these relatively large networks. For comparison purpose, we instead compute the equi- 
librium system throughput that can be achieved in a game if all the MUs are able to connect to all the 
APs at the same time. In such ideal network, all the APs are pooled together as a single "virtual" AP, 
which along with the users form a "virtual" MAC channel. The users can allocate their transmit power 
by computing the NE for the corresponding single AP game, as discussed in Section |lll As suggested 
in Section III-BI when the number of channel becomes large, the throughput achieved by the NE of 
this power allocation game achieves the capacity of the "virtual" MAC. However, we observe that the 
performance of JASPA is close to that of such ideal "multiple-connectivity" network. 

VIII. Conclusion 

In this paper, we addressed the joint AP selection and power allocation problem in a multichannel 
wireless network. The problem was formulated as a non-cooperative game with mixed-integer strategy 
space. We characterized the NEs of this game, and provided distributed algorithms to reach the NEs. 
Empirical evidence gathered from simulation suggests that the quality of the equilibrium solutions is 
reasonably high. 

There can be many future extensions to this work. First of all, the non-cooperative game with mixed- 
integer strategy space analyzed in this paper can be applied to many other problems as well, for example, 
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the downlink counterpart of the current problem. Secondly, for the problem considered in this work, it is 
beneficial to characterize quantitatively the efficiency of the JEP, and to provide solutions for efficiency 
improvement. Thirdly, it would be interesting to investigate the effect of time-varying channels and the 
arrival and departure of the MUs on the performance of the algorithm. 

IX. Appendix 

A. Proof of Proposition |7] 

Note that the WF operator $j(Ij) is also a function of p_,j, hence we can rewrite it as $j(p_j). 
Define *(p) = [*i(p_i), • • • , $Ar(p„Ar)]'''. Define s(p) = *(p) — p. Then the A-IWF algorithm can 
be expressed as: 

p{*+i) = (1 - aW)pW + aW*(pW) = pW + aWs(pW). 
We first state and prove two lemmas. 

Lemma 1: We have s(p)TVpP(p) > M||s(p)|p, ||s(p)|| < Q||VpP(p)||, where M > and Q > 
are two constants. 

Proof: To prove the first inequality, we need to show that: 

N K N K 

J2 E V,.P(p)4Xp) > E E (^'(P))' (30) 

i=l k=l i=l k=l 

where (p) = $J'(p_j) — p^. It suffices to show that for each i G Af, there exists Mj > such that: 

K 

E (Vpf P(P) - A/,s,f (p)) slip) > 0, V z e AA. (31) 

k=l 

Expressing VpfcP(p) explicitly, we have 



,fc|2 



Note that the Lagrangian multiplier ai ensures the tightness of MU i's power budget constraint. Therefore 
we must have XlfcLiPf = Pi^ which in turn implies: 

K K K 

k=l k=l k=l 

Define two index sets ICi = {k : s^(p) > 0}, )C2 = {k : s^(p) < 0}. Then each inequality in (1311 ) is 
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equivalent to 

^ (v,.P(p)-M„s,f(p)),s,f(p) 

> E (^P?^(P) - M^sUp)) (sUp))- (34) 



From (|33] ) we have XlfceA:! '^i^(p) ~ SfceK:2(~'^i (P))- Using this result, we see that to show (l34l ). it 
suffices to show that there exists Mj > such that: 

min{Vp.P(p) - A'UsUp)} > max{V^.F(p) - M^sf (p)}. (35) 

Pick any A;i e /Ci, /c2 G /C2- Below we will show that for the pair {ki,k2) there exists Mf'^'^^'^ > 
such that: 

V^., P(p) - V^.. P(p) > (s^ (p) - s,^^ (p)) . (36) 

Note that if (l36l ) is true, we can take Mi = mirifc^ M^^^'^^\ then (1351) is true, which in turn implies 



(ED- 

In the following, we prove (l36l ) for any pair {ki,k2) with /ci G /Ci and k2 G /C2.Let us first simplify 
the term s^'{p) by denoting ^^^^(p) = [A'^']'^ where A!;^ =ai- ""'^^^^'y^^'' . We can verify 

that: 

< _ pfei = - , V fci e ;*Ci. (37) 

Similarly, the term s(^^(p) can be simplified as s(^^(p) = [^(^^]'*' — pf^- In this case, we have: 

> [4^]+ -p^ > -p^,V fc2 e IC2- (38) 

Due to (l37l ) and (1381 ). in order to prove (l36l ). it suffices to prove that there exists < M^^^''^'^'* < oo 
such that: 

v^., p(p) - Vp.. p(p) > A/i'^'^') ((4^ - p^ ) - - p^ )) . (39) 

Notice that from the definition, for any /c G /C we have — (A^ — Pi) + = ^ fcP(p) - Then the above 
inequality can be simplified to: 

From (I37]l and (l38]l we have that > - (a'I^ -p^) and - (a^^^- - p'iA > 0. Therefore we must have 
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Vpf=iP(p) > Vpfe2P(p). Consequently, (001) is equivalent to M^^^'^'"'^ < v^k,P{p) x \7^k^P{p). Finding 
such M- ^' > is always possible, as VpfcP(p) is bounded away from for any k (cf. (132]) . and note 
> V and n^^ > V A;). 
Now that we can always find a constant M- ' > that satisfies (1361 ). we can take Mj = min^j ^^gx: 



to ensure (|3T| ). Thus, taking M = miui^j^f Mi, (BOl ) is true, and the first part of the proposition is proved. 

The second part of the proposition is straightforward. Due to space limit, we do not show the proof 
here. ■ 

Lemma 2: For two vectors p £ V and p £ V, there must exist constants < D < oo, < K < oo 
such that 

||s(p) - s(p)|| < Z)||p - pII and || VP(p) - VP(p)|| < K\\p - p||. 

Proof: First note that if for all i, there exists a constant < < oo, such that: ||sj(p) — Sj(p)|| < 
^i||P ~ pII' then the first inequality in the lemma is true for D = \/N x maxjg^Z?i. Define a x 1 

vector insrj(p_j) where its kih element is given as — ^' 1^*^ 1"]'^= denote the projection to the 

feasible set Vi. Then from 1261 Lemma 1], we have that $j(p_j) = [— insr^J^.. Notice that we have 



K ^ lLfcl2L„fc .-:k\ 2\ 1/2 



|insrj(p_i) - insri(p_i)j| < ^ 



. fe=i 



k\2 



\ fc=i 



1/2 



<A||p-p|| (41) 
where D, ^ max^ j^^ ff^f | ^JN - 1. Then we have that 

||s,;(p) - Si(p)|l = |l*i(p-,,) - ^^{P-^) + P» - Pi|l 

(a) 

< ||insri(p_i) - insrj(p_i)|| + ||p,, - pi|| 

< aIIp - Pil + Hp. - P.II < (A + i)l|p - pII (42) 

where (a) is because of the triangular inequality and the non-expansiveness of the Euclidean norm. 

Thus, taking Di = Di + 1, we have that ||sj(p) — Sj(p)|| < Di||p — p||. 

The second inequality in the Lemma can be shown similarly. ■ 
Using Lemma [H and Lemma |2l Proposition [T] can be shown by slightly generalizing the existing result 

ll27l Proposition 3.5], which proves the convergence for a family of gradient methods with diminishing 
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stepsizes for unconstrained problems. The generalization of this cited result to the current constrained 
case is to some extent straightforward, and we omit the proof due to space limitations. 

B. Proof of Theorem \3\ 

We first introduce some notations. Define a vector c^*) G such that if hf^ [w] = 1 then c^*) [i] = w. 
Define a set ^ as: ^ = {a : a appears infinitely often in {a^*)}^^}. 

Step 1): We first show that there must exist a NE association profile a* e A. Let us pick any a. & A. 
Suppose a is not a NE association profile, and in iteration T, a^^^ = a. 

From step 3) of the JASPA, for a specific MU i, all the APs w G W^-^^^^ must satisfy ([241). From 
step 2) of the algorithm, for AP a[i], we have P^ji]*^^^ £ £a[{\{^)- From Corollary |2l we have that P~^j^^^ 
must be a NE for the single AP power allocation game G with the set of players {j : a^'^^[j] = a[i]}. 
Utilizing the definition of the NE in we must have 



max Ri I pi^5[. 



,pgf^);Sw)=i?,(pg^^);5w). 



That is, a[i] G yV\ . From step 3) of the algorithm, we see that each w G W^- , in particular a[z], 
has the probability of ^^(j,,,!)^ > -j^ of being selected as c*^^)[i]. From step 4)-step 5) of the algorithm, 
we see that a(^+^)[i] = c(^)[i] with probability at least It follows that 

Probfa(^+i)[z] =a^[z]') > ^ . (43) 

Suppose a is not a NE, then there must exist a MU j such that there is an AP Wj G Wj that satisfies 
Wj / a[j] and 

max [pi,^^ , pi^+^^ ; > Rj (^pJJI^^ ; 

Following the same argument as in the previous paragraph, we have Prob (a(^"'"^)[j] = Wj) > jf^y^/- 
It follows that 



Pr„b(a'-".K.aL'yi)>(-^)" (44, 

The proof of Theorem |2] suggests that if a single MU switches to an AP that increases its rate, then 
the system potential increases. In our current context, this says if a^^"*"^^ = [wj,a^]^j], then P(a(^)) < 
P(a(^+i)). 

Starting from iteration T, with positive probability, in each iteration a single MU switches to its 
preferred AP. The potentials generated in this way is strictly increasing. This process, however, will stop 
at a finite time index T* such that no MU is willing to switch. Consequently, a* = a^* is an equilibrium 
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association profile. The finiteness of T* is from the finiteness of distinctive association profiles (and 
hence the finiteness of possible values for P{a)). Such finiteness combined with the fact that each step 
of of the above process happens with non-zero probability imply that the probability of reaching a* from 
a(^) = a is non-zero. 

We conclude that with non-zero probability, a NE profile a* will appear after a^^^ in finite steps. 
Combined with the assumption that a^^^ = a appears infinitely often, we must also have that a* G A. 

Step 2): We can then show that the sequence {a^*^}^^ converges to an equilibrium profile a*. This 
step can be shown using the same argument as in H] Theorem 2]. Due to space limitation, we choose 
not to reproduce the proof here. 

C. Proof of Proposition |2] 

Proof: For & w ^ W, let = {i : a[i] = w}. Let {in}^^^ be a subsequence in which the subset 
of MUs j\fw is associated with AP w. Clearly, is a subsequence of {t^jj^i- From the J-JASPA 

algorithm, we see that at each in, (|27] ) implements the single AP A-IWF algorithm with the fixed set of 
MUs Mw Therefor, Proposition [J implies that the subsequence |p^f"H (consequently Ipi^"^ ) 

L J n=l L J n=0 

converges to p^, an optimizer of the problem maXp gjr P^„(p^„; a). From Corollary |2] and the fact that 

a(*") = a for all n, we obtain lim„_j.oo Pw ( pS"''; a(*"M = Pw{sl) and lim„_^oo P (p*-*"'', a*^*")) = P(a). 



D. Proof of Proposition \3\ 

Proof: From Proposition |2l we have that for a given a G ^, limn-i-oo Piu""* = ptu^^ which 
implies that lim„^oo I^*^^ = Ij>(a) and lim„^oo I^*^^ = I*,^(S),V i. The latter equality combined with 
the continuity of the rate function R* ^Ij ,„;it;^ with respect to and the continuity of the function 
Ri {py^,w) with respect to p^, further implies that, for any 6 > 0, there must be a constant N{6) such 
that for all n > N{6), the following are true: 



-Ri(P~r-pa[i]) - i?,(pir,;,;a[i] 



<S, 

< 6. (45) 



For any w ^ a[i] satisfying w ^ Bi (I* (a), a), there must exit a > such that: 

R* (I*„(a); «;) - (l*,;,, (a); a^) > e^. (46) 

Take e = min^„gg.(i.(g) ^[i]) ^w, choose a constant 5 satisfying < 25 < e, and let N*{a) = N{6). For 
simplicity of notation, write pf^ instead of Ri fp^*t)j^j,a(*)[i] j. We have that for all n > NI{el), the 
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following is true: 

i?*(l*^(5);«;)-i?,(l*5[,](5);SH 
= Ri w) + R* (if;^ u,) - R: (i[ 

< R* + \r* (lU^);w) ~ R* (1^*;';^.) 

+ |i?f"'-i?,(l*-[,](5);5W) 
<R*(i!l::};w)~R!^"^ + S + d. (47) 



Consequently, we have that for all n > N*{a), R* (lf^z'';u?j - Rf"'' > e - 2? > 0, which impUes that 
w must be in the set Bi a(*")[i]y The claim is proved. ■ 

E. Proof of Theorem |4] 

Proof: Consider the sequence {(a(*\ p^*^)}^]^. Choose a to be any system association profile that 
satisfies: a G argmaxagyi -P(a). Again let |tn : a^*"^ = S and a^*") = a|. From Proposition |2] we have 

that the sequence p*^*") converges, i.e., lim„_s>oo P*-*"-* = P*- We first show that (a, p*) is a JEP. 

Suppose (a, p*) is not a JEP, then there exists a MU i, and a ti) / a[i] such that w £ Bi (l^{a),a[ifj . 
This implies that there exists an e > such that: 

^ (l? JS);*) - Ri > e. (48) 

Define a new association profile sl= [w, a_^]. Following the similar steps as in the proof of Theorem |2j 
we can show that: -P(a) < ^'(a). It is clear that if a G ^, then the previous inequality is a contradiction 
to the assumption that a G argmaXae.4{-P(a)}. In the following, we show that a G ^, which completes 
the proof. 

From Proposition [3j there exists a N-*(a) large enough that for all n > A'? (a), w € Bj (l~^"\a^^"^[ifj . 
Take any n > N'*{a). From the definition, in iteration in, a is sampled, that is, a^*"^ = a. From step 5) 
of the J-JASPA algorithm, we see that with non-zero probability, in iteration in + l, all MUs j i stay in 
a[j], and MU i chooses to switch to w. This implies that the association profile a happens with non-zero 
probability in every time instance t„ + 1. Because {in} is an infinite sequence, a happens infinitely often, 
i.e., a G ^. 

In summary, we conclude that a must be a NE association profile, and thus, (a, p*(a)) is a JEP. 
Finally, following the proofs of Theorem[3l we can show similarly that the sequence {(a^*^ p(a'^*)))}^^ 
generated by the J-JASPA converges to a JEP with probability 1. ■ 
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Proof of Proposition 1 



X: 



I. Proof of Proposition 1 

Proposition 1: If all the MUs in the network employ A-IWF algorithm, then starting from any feasible 
initial power allocations p^^^ G V, their individual power profiles converge to the set of NE of game G 
Proof. To conform to the convention in convex optimization, we define the function -F(p) = — P(p), 



o 

CN ■ and we see that -F(p) is convex. 

^ . Then from the well known Descent Lemma (Lemma 2.1 in HI), and Lemma 2 we have that: 

^ ■ F(p(*+i)) < F(pW) + aWs(pW)TvF(pW) + :|(aW)2||s(pW)|p (1) 

< F(pW) - aWM||s(pW)f + :|(aW)2||s(pW)f 

= F(pW)-aW||s(pW)f (M-aW|). (2) 

From the assumption that a*^*^ goes to 0, there must exist a to such that for all t > to, we have 

^ ■ M — a*^*^^ > 0, and F(p^*^) is monotonically decreasing. Combined with the fact that F(p(*^) is lower 

^1 bounded, then {i^(p^*'')}^i is a convergent sequence. 

■ From ([ill, we have that 

■ 

^ : i^(p(^+^)) < F(p(°)) + E aWs(pW)TvF(pW) + :|(aW)^||s(pW)f . 

t=0 t=0 



It is clear that |[s(p(*))|p is upper bounded, and we have limx^-oo X]t=i('^^*'')^ < so we must have 
limT__!.oo YlJ=o t('^^*^)^I1^(p''*^)IP < Because lim^-^oo -^(p^"^^^^) converges, we must have 



T T 

lim VqWs(pW)TvF(pW) > -oo, or equivalently, lim VaWs(pW)TvP(pW) < oo. (3) 

T->oo ^ — ' T— >oo ■^^^ — ' 
t=0 t=0 

Combining with Lemma 1, we have 

T T 
M lim yQW|[s(pW)f < lim y aWs(pW)TvP(pW) < oo. 

t=0 t=0 

Using the fact that the sequence {a^*^} is not summable, we must have lim inft_j.oo ||s(p^*^)|| = 0. We 
show in the following that in fact we have a stronger result that limf_j.oo ||s(p^*')|| = 0. Suppose not, 
then limsupj_^Qo ||s(p'^*^)|| > 0. In this case there must exist a e > such that the subsequences {r„ : 
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||s(p(-"))|| < e, ||s(p(-"+i))|| > e} and K : e < ||s(pW)|| < 2e,V t G (r„,tx„ - 1), ||s(pK))|| > 2e} 
are both infinite. 

For a specific n, Lemma 2 implies tliat the following is true: 

||s(pK+i))|| _ ||s(p(^"))|| < Z)||p(^"+i) - p(^")|| 

< i?a(^")||s(p(^"))||. (4) 

Thus, there exists a constant N* such that for all n > N*, we must have ||s(p('^"))|| > |. 
We also have the following series of inequalities: 

e < ||s(p(""))|| - ||s(p(""))|| < Z?||p("") - p("")|| 
<D '«^*^l|s(p(*))ll <D '«^*^2e (5) 



which implies 



t=u„ — l 



21) 

t=T„ 

From our previous derivation, we also have lim7^_5.oo X]^o ll^(P^*^) 11^ ^ Then for any 6 > 
there must exists a N{6) such that for all n > iV(5): Ylt=T'^^^ ||s(p(*)) |p < (5. 
Take 5 = and take n > max | A^*, iV(^)|, then we have 

2 t=u„ — l t=u„ — l 2 

7 E E "^*^ii«(p^*^)f (7) 

i=T„ t=T„ 

which implies X]*=r"~^ — 2I) • This is a contradiction to (lU. Thus, we conclude that lim sup^.^.^^ ||s(p^*^ ] 
0, and consequently lim^^^oo l|s(p''*-')|| = 0. 

From lim(_>,oo ||s(p^*^)|| = we see that the limit point p* of any converging subsequence of {p*^*^}, 
must satisfy $(p*) = p*, which ensures that p* is a NE of the game G. Consequently p* must maximize 
the function P(p) (from Corollary 1), and this implies that the entire sequence {P{p^^^)}'^i converges 
to the value maxpgp ^'(p)- 
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